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I. Basis of the report 

1 . With regard to the elements of the international application (Replacement sheets which have been furnished to 
the receiving Office in response to an invitation under Article 14 are referred to in this report as "originally filed" 
and are not annexed to this report since they do not contain amendments (Rules 70. 16 and 70.17)): 



Description, Pages 

1 , 4, 5, 7-28 as originally filed 

2, 3, 6 received on 03.01 .2005 with letter of 03.01 .2005 
Claims, Numbers 

1 -46 received on 03.01 .2005 with letter of 03.01 .2005 



Drawings, Sheets 

1/12-12/12 as originally filed 

2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under 
Rule 55.2 and/br 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure 
in the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 

□ the description, pages: 

El the claims, Nos.: 47,48 

□ the drawings, sheets: 
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5. □ This report has been established as if (some of) the amendments had not been made, since they have 

been considered to go beyond the disclosure as filed (Rule 70.2(c)). 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 

6. Additional observations, if necessary: 
IV. Lack of unity of invention 

1 . In response to the invitation to restrict or pay additional fees, the applicant has: 

□ restricted the claims. 

□ paid additional fees. 

□ paid additional fees under protest. 

□ neither restricted nor paid additional fees. 

2. S This Authority found that the requirement of unity of invention is not complied with and chose, according to 

Rule 68.1 , not to invite the applicant to restrict or pay additional fees. 

3. This Authority considers that the requirement of unity of invention in accordance with Rules 13.1 , 13.2 and 13.3 
is 

□ complied with. 

H not complied with for the following reasons: 
see separate sheet 

4. Consequently, the following parts of the international application were the subject of international preliminary 
examination in establishing this report: 

IS all parts. 

□ the parts relating to claims Nos. . 

V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 



Novelty (N) 


Yes: 


Claims 


1-46 




No: 


Claims 




Inventive step (IS) 


Yes: 


Claims 


1-46 




No: 


Claims 




Industrial applicability (IA) 


Yes: 


Claims 


1-46 




No: 


Claims 





2. Citations and explanations 
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IV Lack of unity of invention 

1 Independent claims 1 and 34 are both directed to a hand-held device comprising a 
processor, a display, a user interface, a means for sensing motion of the hand-held 
device, and means for transforming the sensed motion into a signal suitable for 
controlling the hand-held device in some way. This subject matter common to claims 1 
and 34 is known from 

WO 01/86920 

figure 1 of which clearly shows a hand-held device having a display and a user interface. 
The passage at lines 15-31 on page 5 clearly discloses using accelerometers or a 
camera to produce a signal indicative of movement of the hand-held device, and this 
signal is clearly used for controlling the hand-held device. Figure 6 shows the processor 
(602) of the hand-held device. 

Apart from the above common features the features of claims 1 and 34 are quite distinct, 
claim 1 including a digital camera for producing images to be presented on the display, 
and claim 34 including a graphical user interface with objects and a cursor, whereby the 
signal indicative of motion is used to control movement of a cursor over the display. 
Claims 1 and 34 therefore fall to be considered as relating to two different inventions. 

2 Independent claims 32 and 33 are considered to relate to the same invention as claim 
1 , though it may be that slight revision of both claims 32 and 33 is required to make quite 
clear that the images produced by the digital camera are intended to be presented on 
a display forming part of the hand-held device. 

3 No great extra effort is required of the examiner in assessing the patentability of the 
claims relating to both inventions - this will therefore be done. 

V Reasoned statement under Article 35(2) 

1 First invention (claims 1-33) 

1 .1 Claim 1 requires that the digital camera produces images which are presented on the 
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display of the hand-held device, and the output signal of the camera is used to derive 
a signal indicative of movement of the hand-held device, this movement-indicative signal 
then being employed as user input for controlling the hand held device. There is prior art 
(e.g. WO 01/86920) disclosing a hand-held device including a camera which produces 
a movement-indicative signal used for controlling the hand-held device, but there is no 
prior art disclosing or suggesting the use of a camera (on a hand-held device) for 
producing both images for display and also movement-indicative signals used for 
controlling the hand-held device. The subject matter of claim 1 is thus considered to be 
new and to have inventive step. 

1 .2 The same conclusion holds for- independent claims 32 and 33 as notionally amended for 
consistency with claim 1 (see IV. 2 above), and also for dependent claims 2-31 . 

2 Second invention 

2.1 There is prior art, e.g. WO 01/86920, disclosing all the features of the first five lines of 
claim 34, but in the prior art the movement-indicative signal is used for scrolling the 
display or for achieving a zooming function. There is no disclosure or suggestion in the 
prior art of using a movement-indicative signal for causing movement of a cursor across 
the display. The subject matter of claim 34 is thus considered to be new and to have 
inventive step. 

2.2 The same conclusion holds for dependent claims 35-46. 

3 The claimed invention finds industrial applicability in the technical field of hand-held 
devices such as mobile telephones. 
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commands, e.g. changing the area viewed oh the display screen 
or controlling the 'per f biro^ce " of"" a"* "particular parameter 
associated with the device. Further, given the limited area 
available, not only on the display screen but also on the 
5 entire device, adding additional control keys, etc., is both 
difficult and burdensome to a user requiring two hand 
operation of the device. 



US 6466198 discloses a system and method for view navigation 
10 and magnification of the display of hand-held devices in 
response to the orientation changes along only two axes of 
rotation as measured by sensors inside the devices. The view 
navigation system can be engaged and controlled by 
simultaneously pressing switches . on both sides of the hand- 
15 held device • Miniature sensors like accelerometers, tilt 
sensors, or magneto-resistive direction sensors sense the 
orientation changes. These miniature sensors are presently not 
typically standard equipment for hand-held devices. Thus, such 
sensors add cost, use precious space and add weight. 



20 



25 



The present invention is directed toward overcoming one or 
more of the above- identified problems. 

DISCLOSURE OF THE INVENTION 



On this background, it is an object of the present invention 
to provide a hand-held device of the kind referred to 
initially, which allows user input with the same hand that 
holds the device, without requiring the dedicated sensatory 
30 equipment used by prior art hand- held devices. 

This object is achieved in accordance with claim 1, by 
providing a hand-held device comprising: a processor, a 
display, a digital camera for capturing motion video or still 
35 images, means for . presenting still images or motion video 
captured by said digital camera on said display ; means • for 
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• transforming a signal from the - camera into a . motion* signal 
indicative of the motion of the hand-held device; and a user 
interface in which motion of the hand-held device is - through 
the motion signal derived thereof - used as a us er input to 

5 control operation of the hand-held device. 

Thus, by using a sensor that is available to start with in 
many hand-held devices — namely a digital camera — for a 
secondary use, namely creating a motion signal indicative of 
10- the motion of the hand-held device, a hand-held device with 
motion sensing is provided in a economical and reliable 
manner . 

The hand-held device may further comprise a user interface in 
15 which motion of the hand-held device is - through the motion 
signal derived thereof - used as a user input. 

The hand-held may further comprise a display, preferably a 
display suitable for displaying capt-ured images. 

20 

Motion of a given type of the hand- held device can be used to 
manipulate images shown at least in part on the display, 
preferably by moving the images in a manner substantially 
corresponding to the movement of the hand-held device. 

25 

• Different types of motion the hand-held device can e.g. be 
used to move, and/or zoom, and/or expand/ collapse and/or 
rotate images displayed on the display. 

30 Motion substantially parallel to the plane of the display of 
the hand-held device can be used to scroll an image displayed 
on the display. Motion substantially perpendicular to the 
plane of the display can be used to zoom an image displayed on 
the display. Rotational motion of the hand-held device can be 

35 used to rotate an image displayed on the display. 
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interface, and the functionality of the second key can be 
associated with calling up a context-sensitive menu, 

5 The selection of the object concerned can be performed by 
pressing and releasing the first key. Activation of the object 
concerned can be performed by pressing and releasing the first 
key twice in rapid succession. Moving or resizing of the 
object concerned can be performed by holding down the first 

10 key while moving the hand- held device to move the cursor. 

The first key and the second key can be softkeys, whereby the 
current functionality of the softkeys is shown in the display, 
preferably in dedicated fields of the display. 



.' The first key can be placed below the display on the left side 
of the latter, preferably proximate to lower edge of the 

• display, and the second key can be placed below the display on 
the right side of the latter, preferably proximate to lower 
20 edge of the display. 

It is another object of the present invention to provide an 
improved method for proving user input to hand-held devices. 
This object is achieved by providing a method for creating 

25 user input for a hand-held device that has a processor, a user 
interface and a digital camera for capturing motion video or 
still images comprising the steps of: obtaining a camera 
signal, alternatingly using the, camera signal to capture still 
Images or motion video; or to d etermine motion of the hand- 

30 held device from the camera signal; and using the determined 
motion of the hand-held device as an input for the user 
interface . 

It is yet another object of the present invention to provide a 
35 use of a digital camera of a hand-held device « 



15 
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CLAIMS: 

1. A hand-held device comprising: 
5 a processor; 

a display; 

a digital camera for capturing motion video or still 
10 images; 

means for presenting still images or motion video 
captured by said digital camera on said display; 

15 means for transforming a signal from the camera into a 

motion signal indicative of the motion of the hand-held 
device; and 

. a user interface in which motion of the hand-held device 
20 is - through the motion signal derived thereof - used as 

a user input to control operation of the hand-held 
device . 

2- A hand-held device according to claim 1, in which motion of 
25 a given type of the hand-held device is used to manipulate 
images shown at least in part on the display, preferably by 
moving the images in a manner substantially corresponding to 
the movement of the hand-held device, 

30 3. A hand-held device according to claim 2, in which a given 
type of motion the hand- held device is used to move, and/or 
zoom, and/or expand/collapse and/or rotate images displayed on 
the display- 

35 4. A hand-held device according to claim 3, in which motion 
substantially parallel to the . plane of the . display of * the . 
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hand-held device is used to scroll an. image displayed on the • 
display, and/or motion substantially perpendicular to the 
plane of the display is used to zoom an image displayed on' the 
display and/or rotational motion of the hand-held device is 
5 used to rotate an image displayed on the display. 

5. A hand-held device according to any of claims 2 to 4, in 
which the images are images previously captured by the camera. 

10 6. A hand-held device according to any of claims 2 to 5, in 
which movement of image is inverted with respect to motion of 
the hand-held device. 

7. A hand-held device according to any of claims 1 to 6, in 
15 which the user interface comprises a graphical user interface, 

. and wherein motion of the hand-held device is used as an input • 
to the graphical user interface. 

8. A hand-held, device according to claim 2, in which motion of 
20 the hand-held device is used to manipulate an object displayed 

by the graphical user interface, preferably by moving the 
object in a manner substantially corresponding to the motion 
or to the inverted motion of the hand-held device, whereby the 
object displayed by the graphical user interface can be, an 
25 icon, a dialogue box, a window, a menu or a pointer. 

9. A hand-held device according to claim 7, in which motion of 
a given type of the hand- held device is used to move, and/or 
zoom, and/or expand/collapse and/or rotate objects displayed 

30 by the graphical user interface. 

10. A hand-held device according to claim 9, in which motion 
substantially parallel to the plane of the display of the 
hand-held device is used to scroll an object displayed by the 

35 graphical user interface, . and/or motion substantially 
perpendicular to the plane of the display is used to zoom an 
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• object displayed by the graphical user interface and/or 
rotational motion of the hand-held device is used to rotate an 
object displayed by the graphical user interface. 

5 11. A hand-held device according to any of claims 1 to 10, in 
which the digital camera is detachable > 

12. A hand-held device according to any of claims 1 to 11, in 
which the digital camera is movable relative to the hand-held 
10 device . 



13. A hand-held device according to any of claims 1 to 12, in 
which the means for transforming a signal from the camera into 
a motion signal derives the motion signal from changes between 

15 succeeding images, or parts of succeeding images captured by 
the camera. 

14. A hand-held device according to any of claims 1 to 13, in 
which the camera has an autof ocus system, whereby the focusing 

20 setting of the autofocus system is used for detecting movement 
in the camera direction. 

• 15. A hand-held device according to any of claims 1 to 14, 
further comprising at least one key, wherein the functionality 
25 of a motion type is dependent on the state of the at least one 
key. 

16. A hand-held device according to any of claims 1 to 15, in 
which rotational motion of the hand-held device about an axis 

30 substantially perpendicular to the display results in an 
inverse rotational movement of the image or graphical user 
interface object relative to the display, preferably in a 
manner such that the image or object is static with respect to 
the fixed coordinate system in which the hand-held, device is. 

35 situated. 
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15. 



17. A hand- held device according to' any of claims 1 to 16 in 
which the motion signal is used to adjust device settings, the 
device settings preferably comprising sound settings and 
display settings. 



18. A hand-held device according to any of claims 7 to 17, 
further comprising a keypad with at least a first- and a 
second key and the graphical user interface comprises a 
cursor, whereby motion of the hand-held device is used to 
10 position the cursor over an object of the graphical user 
interface and primary functions associated with the object 
concerned are activated by pressing the first key and 
secondary functions associated with the . object of the 
concerned are activated by pressing the second key. 

19. A hand-held device according to . claim 18, in which the 
functionality of the first key is associated with selection 
and activation of objects of the graphical user interface, and 
in which the functionality of the second key is preferably 

20 associated with calling up a context-sensitive menu. 

20. A hand- held device according to claim 19, in which 
. selection of the object concerned is performed by pressing and 

releasing the first key, and activation of the object 
25 . concerned is preferably performed by pressing and releasing 
the first key twice in rapid succession. 

. 21. A hand-held device according to claim 13 or 20, in which 
moving or resizing of the object concerned is performed by 
30. holding down the first key while moving the hand-held device 
to move the cursor. 

22. A hand-held device according to any of claims 18 to 21, in 
which the first key and the second key are softkeys whereby 
35 the current functionality of the softkeys is shown in the 
display, preferably in dedicated fields of the display. 
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23. A hand-held device according to claim 22, in which the 
first key is placed below the display on the left side of the 
latter, preferably proximate to lower edge of the display, and 

5 the second key is placed below the display on the right side 
of the latter, preferably proximate to lower edge of the 
display* 

24. A hand-held device according to any of claims 1 to 23, 
10 further comprising at least one gravity based tilt sensor, and 

whereby the signal from the at least one tilt sensor is used 
in combination with the signal from the camera for creating 
the motion signal. 

15 25. A hand-held device according to claim 24, wherein a tilt 
sensor is associated with the X-axis and/or a tilt sensor is 
associated with the Z-axis. 

26. A hand-held device according to claim 25, wherein the 
20 signal form the at least one tilt sensor is used to determine 

the absolute orientation of the handheld device relative to 
the direction of the gravitational pull. 

27. A hand-held device according to any of claims 1 to 26, 
25 further comprising means for sending the motion signal to 

another terminal via cable, infrared waves or radio frequency 
• waves. 

28. A system comprising a hand-held device according to claim 
30 27 and a terminal capable of displaying imaginary three- 
dimensional objects on a two-dimensional screen, said terminal 
comprising means to change the orientation of the displayed 
object in response to signals received from the handheld 
device, whereby orientation changes of the hand-held device 

35 are translated to corresponding orientation changes-, of the 
displayed object. 
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29. A system according to claim 28, in which position changes 
of the handheld device are translated to position changes of 
the displayed object. 

5 

30. A system comprising a hand-held device according to claim 
27 and a terminal capable of displaying an imaginary three- 
dimensional space on a two-dimensional screen, said terminal 
comprising means to change the viewing position in the 

10 imaginary three-aimensional space in response to signals 
received from the handheld device, whereby positional changes 
. of the hand- held device are translated to corresponding 
changes in the viewing position. 

15 31. A system according to claim 28, in which orientation 
changes of the handheld device are translated into 
corresponding changes in the viewing direction in the 
. imaginary three-dimensional space. 

20 32. A method for creating user input for a hand-held device 
that has a processor, a user interface and a digital camera 
for capturing motion video or still images comprising the 
steps of: 

obtaining a camera signal; 
25 altematincrlY using the camera signal to: 

capture still images or motion video; or 
to determine motion of the hand-held device from 
the camera signal; and to 

use the determined motion of the hand-held device as an 
30 input for the user interface. 

33. Use of a digital camera In a hand-held device for both 
capturing motion video or still images and for producing a 
motion signal indicative of motion of the hand-held device - 
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15 



34. A hand-held device comprising a processor, means for • 
sensing motion of the hand-held device, a display, a keypad 
with at least a first- and a second key, a graphical user 
interface with objects and a cursor, and means for 
transforming the sensed motion of the handheld device into a 
signal suitable for moving the cursor over the display. 

35. A hand-held device according to claim 34, in which motion 
of the hand-held device is used to position the cursor over an 
object of' the graphical user interface and primary functions 
associated with the object concerned are activated by pressing 
the first key and secondary functions associated with the 
object concerned are activated by pressing the second key. 

.36. A hand-held device according to claim 35, in which the 
•functionality of the first key is associated with selection 
and activation of objects of the graphical user interface, and 
in which the functionality of the second key is preferably 
associated with calling up a context-sensitive menu. 



20 



37. A hand-held- device according to claim 36, in which 
selection of the object concerned is performed by pressing and 
releasing the first key, and activation of the object 
concerned is preferably performed by pressing and releasing 

25 the first key twice in rapid succession. 

38. A hand-held device according to claim 36 or 37, in which 
moving or resizing of the object concerned is performed by 
holding down the first key while moving the hand-held device 

30 to move the cursor and the object concerned in unison 
therewith. 

39. A hand-held device according to any of claims 34 to 38, in 
which the first key and the second key are softkeys whereby 

35 the current functionality of the softkeys is shown in the 
display, preferably in dedicated fields of the display. 
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.40. A hand-held device according to claim 39 , in which the 
first key is placed below the display on the left side of the 
latter, preferably proximate to lower edge of the display, and 
5 the second key is placed below the display on the right side 
of the latter, preferably proximate to lower edge of the 
display. 

41. A hand-held device according to any of claims 34 to 40, in 
10 which said means for transf orming motion of the handheld 
device into a signal suitable for moving the cursor over the 
display comprises a tilt sensor and/or an image capturing 
device and/ or an accelerometer . 

15 42. A hand-held device according to claim 41, in which said 
image capturing device is a motion video or still image 
digital camera. 

43. A hand-held device according to any of claims 34 to 42, 
20 further comprising means to transform a signal from the camera 
and/or tilt sensor and/or accelerometer into a position signal 
for the cursor. 

.44. A hand-held device according to claim 43, in which said 
25 means for transforming a signal from the camera into a motion 
signal derives the motion signal from changes between 
succeeding images, or parts of succeeding images captured by 
the camera. 

30 .45. A hand-held device according to any of claims 42 to 44, in 
which the camera has an autofocus system, whereby the focusing 
setting of the autofocus system is used for detecting movement 
in the camera direction. 

35. 46. A hand-held device according to any of claims 34 to 45 , in 
which the graphical user interface includes one or more of the 



° 0128 ~ PCT AMENDED SHEET 
r- _ / • , mars /ai /none nr>« ac\ 'Okk D niA 



03-01 -20.05^ 9:51 FAX T02DI.3T W1P — BPO MHUCH _ ^0300607 



# 




36 



following object types: icons, dialogue boxes*, windows, menus, 
pointers. 
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